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Abstract 

Suppose we have three independent copies of a regular diffusion on [0,1] with 
absorbing boundaries. Of these diffusions, either at least two are absorbed at the 
upper boundary or at least two at the lower boundary. In this way, they determine 
a majority decision between and 1. We show that the strategy that always runs 
the diffusion whose value is currently between the other two reveals the majority 
decision whilst minimising the total time spent running the processes. 
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1 Introduction 

Let Xi,X% and A3 be three independent copies of a regular diffusion on [0, 1] with ab- 
sorbing boundaries. Eventually, either at least two of the diffusions are absorbed at the 
upper boundary of the interval or at least two are absorbed at the lower boundary. In 
this way, the diffusions determine a majority decision between and 1. 

In order to identify this decision we run the three processes -not simultaneously, but 
switching from one to another- until we observe at least two of them reaching a common 
boundary point. Our aim is to switch between the processes in a way that minimises the 
total time required to find the majority decision. 

More precisely, we allocate our time between the three processes according to a suit- 
ably adapted [0, oo) 3 -valued increasing process C with X^=i^W = t- Such a process is 
called a strategy and Ci(t) represents the amount of time spent observing Aj after t > 
units of calendar time have elapsed. Accordingly, the process we observe is 

X c d ^ f (X 1 (C 1 (t)),X 2 (C 2 (t)),X 3 (C 3 (t));t > 0), 

and the decision time r c for the strategy C is the first time that two components of X 
are absorbed at the same end point of [0, 1], i.e. 

r c = inf{t > : Af (t) = Xf(t) G {0, 1} for distinct 



In this paper we find a strategy C* that minimises this time. Roughly speaking, C* 
runs whichever diffusion is currently observed to have "middle value" (see Lemma fTTil for 
a precise description). Our main theorem is that the decision time r c of this strategy is 
the stochastic minimum of all possible decision times, i.e. 

Theorem 1.1. The decision time t c of the "run the middle" strategy C* given in lemma 



l.J\ satisfies 



P(r c * > t) = inf P(t c > t), for every t > 0. 

where the infimum is taken over all strategies and t c is the corresponding decision time. 

This model fits into the existing literature on optimal dynamic resource allocation 
(see 11.11 below for a brief review) but our original motivation for studying it was to gain 
an understanding of the problem of evaluating the "recursive majority of three" function 
on random input. The latter can be described as follows - take the complete ternary 
tree on n levels, place independent Bernoulli (p) (0 < p < 1) random variables on each 
of the 3 n leaves and recursively define the internal nodes to take the majority value of 
their children. We wish to determine the value of the root node, but may only accrue 
knowledge about the tree by sequentially observing leaves, paying £1 each time for the 
privilege. It remains an open problem to determine the strategy with least expected cost, 
r n . However, r is sub-multiplicative (i.e. r n+m < r n r m for any n, m £ N) and so, by 
Fekete's lemma 
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exists with the trivial bounds 2 < 7 < 3. The value of 7, despite attracting the atten- 
tion of several investigators, is not known (see section 16. 2j) . Our idea was to study it 
by considering a continuous approximation to the large n tree. It was this continuous 
approximation that inspired the diffusive model introduced in this paper, but the reader 
is warned that the results we present here do not shed light on the value of 7. 

However, the problem of switching between diffusions is worthwhile in its own right. 
It has a similar flavour to the continuous multi-armed bandit problem but does not seem 
to have the same mathematical anatomy. Nevertheless there is an interesting structure 
to be revealed - in particular we make use of the heuristic equation (I2.17P in order to 
evaluate the value function for the discounted problem, and the same equation plays a 
central role in proving the much stronger stochastic minimality property. 



1.1 Dynamic resource allocation 

The problem we have described is one of optimal dynamic resource allocation in continu- 
ous time. The most widely studied example of this is the continuous multi-armed bandit 
problem (see, for example, El Karoui and Karatzas [6], Mandelbaum and Kaspi [12J). 
Here, a gambler chooses the rates at which he will pull the arms on different slot ma- 
chines. Each slot machine rewards the gambler at rates which follow a stochastic process 
independent of the reward processes for the other machines. These general bandit prob- 
lems find application in several fields where agents must choose between exploration and 
exploitation, typified in economics and clinical trials. An optimal strategy is suprisingly 
easy to describe. Associated to each machine is a process known as the Gittins index, 
which may be interpreted as the equitable surrender value. It is a celebrated theorem 
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that at each instant, we should play whichever machine currently has the largest Gittins 
index. This is in direct analogy to the discrete time result of Gittins and Jones [8]. 

There is no optimal strategy of index type for our problem. This reflects the fact that 
the reward processes associated to running each of the diffusions are not independent - 
once two of the diffusions are absorbed, it may be pointless to run the third. 

In [16], a different dynamic allocation problem is considered. It has a similar flavour 
in that one must choose the rates at which to run two Brownian motions on [0, 1], and 
we stop once one of the processes hits an endpoint. The rates are chosen to maximise 
a terminal payoff, as specified by a function defined on the boundary of the square (the 
generalisation of this problem to several Brownian motions is considered in [5T]). An 
optimal strategy is determined by a partition of the square into regions of indifference, 
preference for the first Brownian motion and preference for the second. However, there 
is no notion of a reward (cost) being accrued as in our problem. So, our problem, in 
which time is costly and there is a terminal cost of minus infinity for finishing on a part 
of dS which does not give a majority decision could be seen as lying between continuous 
bandits and the Brownian switching in [15] . 

1.2 Overview of paper 

The rest of the paper is laid out as follows. Section 11.31 contains a precise statement of 
the problem and our assumptions and a clarification of Theorem 11.11 The proof of this 
theorem begins in Section 2, where we show that the Laplace transform of the distribution 
of the decision time r c solves certain differential equations. This fact is then used in 
Section 3 to show that the tail of r c solves, in a certain sense, the appropriate Hamilton- 
Jacobi-Bellman PDE. From here, martingale optimality arguments complete the proof. 
Section 4 shows the existence and uniqueness of the strategy C* and in Section 5 we 
explain the connection between the controlled process and doubly perturbed diffusions. 
In the final section, we make a conjecture about an extension to the model and then, to 
close, we ask a few questions relating to the discrete recursive majority of three problem 
that motivated us originally. 

1.3 Problem statement and solution 

We are given a complete probability space (fl, J 7 , P) supporting three independent Ito dif- 
fusions (Xi(t), t > 0), i G V — {1, 2, 3}, each of which is started in the unit interval [0, 1] 
and absorbed at the endpoints. The diffusions all satisfy the same stochastic differential 
equation 

dXi{t) = <T{Xi(t))dBi{t) + v(Xi(t))dt, t > 0, (1.1) 

where a : [0, 1] — > (0, oo) is continuous, /x : [0, 1] — *• R is Borel and (Bi(t), t > 0), i G V 
are independent Brownian motions. 

We denote by S the unit cube [0, l] 3 , by R + the set of non-negative real numbers [0, oo) 
and ^ its usual partial order on R 3 ^. It is assumed that we have a standard Markovian 
setup, i.e. there is a family of probability measures (F x ] x G S) under which X(0) = x 
almost surely and the filtration Ti = (J-i(t)\t > 0) generated by Xi is augmented to 
satisfy the usual conditions. 
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From here, we adopt the framework for continuous dynamic allocation models pro- 
posed by Mandelbaum in [15] . This approach relies on the theory of multiparameter time 
changes; the reader may consult Appendix A for a short summary of this. 

For T) G we define the a-algebra 

T(rf) = a{Ti{r]i), ^(^j^sfe)), 

which corresponds to the information revealed by running X^ for rji units of time. The 
family (J 7 (t]);t] G K.1) is called a multiparameter filtration and satisfies the "usual con- 
ditions" of right continuity completeness and property (F4) of Cairoli and Walsh [2]. 
It is in terms of this filtration that we define the sense in which our strategies must be 
adapted. A strategy is an IR^-valued stochastic process 

C = (C 1 (t),C 2 (t),C 3 (t);t>0) 

such that 

(CI) for i = 1,2, 3, Cj(0) = and Cj(-) is nondecreasing, 
(C2) for every t > 0, C x (t) + C 2 {t) + C 3 (t) = t and 

(C3) C(t) is a stopping point of the multiparameter filtration (^(rf); i] G R 3 + ), i.e. 

{C(t) ^ 7]} G J 7 (if) for every 77 G R+. 

Remark 1.2. In the language of multiparameter processes, C is an optional increasing 
path after Walsh [22] . 

Remark 1.3. Conditions (CI) and (C2) together imply that for any s <t, \Ci(i) —Ci(s)\ < 
t — s. It follows that the measure dCi is absolutely continuous and so it makes sense to 
talk about the rate Ci(t) = dCi(t)/dt, t > 0, at which Xj is to be run. 

The interpretation is that Ci(t) models the total amount of time spent running Xi by 
calendar time t, and accordingly, the controlled process X c is defined by 

X c (t) d = f (X 1 (C 1 (t)),X 2 (C 2 (t)),X 3 (C 3 (t))), t > 0. 

Continuity of C implies that X c is a continuous process in S that is adapted to the 
(one parameter) filtration T c defined by 

T c (t) = {F EF: FD {C(t) ^r]} G F(r]) for every 77 G , t > 0. 

The decision time r c for a time allocation strategy C is the first time that X c hits 
the decision set 

D = f {(xi, x 2 , x 3 ) G S : Xj = Xj G {0, 1} for some 1 < 2 < j < 3} 

The objective is to find a strategy whose associated decision time is a stochastic 
minimum. Clearly, it is possible to do very badly by only ever running one of the processes 
as a decision may never be reached (these strategies do not need to be ruled out in our 
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model). A more sensible thing to do is to pick two of the processes, and run them untill 
they are absorbed. Only if they disagree do we run the third. This strategy is much 
better than the pathological one (the decision time is almost surely finite!) but we can 
do better. 

We do not think it is obvious what the best strategy is. In the situation that Xi(0) 
is close to zero and X^(0) close to one, it is probable that X\ and X3 will be absorbed at 
different end points of [0, 1]. So, if X 2 (0) is close to 0.5 say, it seems likely that X 2 will 
be pivotal and so we initially run it, even though X± and X3 might be absorbed much 
more quickly. Our guess is to run the diffusion whose value lies between that of the other 
two processes. But if all the processes are near one, it is not at all clear this is the best 
thing to do. For example, one could be tempted to run the process with largest value in 
the hope that it will give a decision very quickly. 

It turns out that we must always "run the middle". That is, if, at any moment t > 0, 
we have Xf (t) < X%(t) < X3 (£), then we should run X 2 exclusively until it hits Xf (i) or 
Xjftt). We need not concern ourselves with what happens when the processes are equal. 
This is because there is, almost surely, only one strategy that runs the middle of the three 
diffusions when they are separated. To state this result, let us say that for a strategy C, 
component C\ increases at time t > if Ci(u) > Ci(t) for every u > t. 

Lemma 1.4. There exists a unique time allocation strategy C* such that C* increases only 
at times t > such that Xf (t) < Xf * (t) < Xf (t) under some labelling {i,j,k} = V of 
the processes. 

If C is any other strategy with this property, then C(t) = C*(t) for all t > almost 
surely (with respect to any of the measures W x ). 

This lemma is proved in section H] and Theorem 11.11 states that C* gives a stochastic 
minimum for the decision time. 

In the sequel, the drift term fi is assumed to vanish. This is not a restriction, for if a 
drift is present we may eliminate it by rewriting the problem in natural scale. 

2 The Laplace transform of the distribution of t c * 

The proof of Theorem 11.11 begins by computing the Laplace transform 

v r (x) d = E x (exp(-rr c *)) , 

of the distribution of the decision time. This non-trivial task is carried out using the 
"guess and verify" method. Loosely, the guess is inspired by comparing the payoffs 
of doing something optimal against doing something nearly optimal. This leads to a 
surprisingly tractable heuristic equation from which v r can be recovered. 

The argument which motivates the heuristic proceeds as follows. From any strategy 
C it is possible to construct (but we omit the details) another strategy, C, that begins 
by running X± for some small time h > (i.e. C(t) = (t, 0, 0) for < t < h) and then 
does not run X\ again until C± exceeds h, if ever. In the meantime, C 2 and C3 essentially 
follow C 2 and C3 with the effect that once C\ exceeds h, C and C coincide. 

This means that if the amount of time, Ci(r c ), that C spends running Xi is at least 
h, then r c and r c are identical. On the other hand, if C\(t c ) < h, then C runs X\ for 
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longer than C, with some of the time C spends running X\ being wasted. In fact, outside 
a set with probability o(h) we have 

r d = r c + (/i-T 1 ) + , (2.2) 

where T, = C;(t c ) is the amount of time that C spends running Xi while determining the 
decision. 

We compare C with the strategy that runs X\ for time h and then behaves optimally. 
If we suppose that C* itself is optimal and recall that v r is the corresponding payoff, this 
yields the inequality 

Ex (exp(-rr d )) < E x (exp(-r/i)^(X 1 (/ i ), X 2 (0), X 3 (0))) . (2.3) 

Now, we take C = C* and use (12. 2p to see that the left hand side of (I2.3P is equal to 

E x (exp(-r(r c * + (h - 7\) + ))) + o{h), 
which, in turn, may be written as 

v r (x) + E x ((exp(-r(r c * + h) - exp(-rr c *)) 1 [T . =0] ) + o(h). (2.4) 
On the other hand, if we assume v r is suitably smooth, the right hand side of (12.31) is 
v r (x) + h(Q 1 - r) v r (x) + o{h), Xl G (0, 1). (2.5) 
where we have introduced the differential operator Q l defined by 

^f{ x )^ l -a\ Xi )^f{x), x 4 G(0,l). 

After subsituting these expressions back into (12. 3p and noticing that there was nothing 
special about choosing X\ to be the process that we moved first, we see that 

E x (exp(-r(r c * + h) - exp(-rr c *);^, = 0) < h (Q i - r) v r (x) + o{h), (2.6) 

for each G (0, 1) and % G V. 

Dividing both sides by h, and taking the limit h — > yields the inequality 

(Q i - r) v r {x) < -rE, (exp(-rr c *); T { = 0) . (2.7) 

Now, in some simpler, but nevertheless related problems, we can show that (12. 7p is 
true with an equality replacing the inequality. This prompts us to construct a function 
satisfying (12. 7p with equality. Our effort culminates in 

Lemma 2.1. There exists a continuous function h r : S — > M such that 

• h r (x) = 1 for x G D, 

• the partial derivatives q^ H q x exist and are continuous on {x G S\D : x«, Xj G (0, 1)} 
(for any i,jEV not necessarily distinct) and 
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• furthermore, for each i G V and x D with Xi G (0, 1), 

(gt-r) h r (x) = -rftx), 

where fyx) = E x (exp(-rr c *) l[ Ti =o]) ■ 

Proof. We begin by factorising f^{x) into a product of Laplace transforms of diffusion exit 
time distributions. This factorisation is useful as it allows us to construct h by solving a 
series of ordinary differential equations. Note that in this proof, we will typically suppress 
the r dependence for notational convenience. 

The diffusions all obey the same stochastic differential equation and so we lose nothing 
by assuming that the components of x satisfy < x\ < X2 < x 3 < 1. Further, we suppose 
that x D because otherwise Tj = P^-almost-surely. 

In this case, T 2 > P^-almost surely, because for any t > 0, there exist times t 1: t 3 < 
t/2 at which Xi(ti) < x 1 < x 2 < x 3 < X 3 (t 3 ) and so it is certain our strategy allocates 
time to X 2 . It follows that f 2 (x) vanishes. 

Now consider f 1 . There is a P x -neglible set off which T\ — occurs if, and only if, 
both of the independent diffusions X 2 and X 3 exit the interval (Xi(0),l) at the upper 
boundary. Furthermore, r c * is just the sum of the exit times. That is, if 

mi i} = inf {t > : X^t) = a}, a G /, i G V, (2.8) 

then 

f\x) = E x (exp(-r(mS 2) +mf } )) \ m ^ <m ^^ <m ^) ■ 
Using independence of X 2 and X 3 , we have the factorisation 

3 

f 1 (x) = l[E x (exp(-rmP) l [m « <m «,) . 

Note that our assumption x ^ D guarantees that X\ < 1. 

To write this more cleanly, let us introduce, for < a < b < 1, the functions 

K, b (u)=E u (expt-rm^m^mW), 

where the expectation operator E« corresponds to the (marginal) law of X 1 when it begins 
at u G [0, 1]. 

The diffusions obey the same SDE, and so 

Similarly, 

f 3 (x) = h 0tX3 ( Xl )h^ X3 (x 2 ) 

where 

^» = E U (exp(-rm«);m«<m«). 
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We take, as building blocks for the construction of h, the functions h 01 , abbreviated 
to in the sequel. The regularity of each of our (non- singular) diffusions together with 
the Markov property shows that if a < b, u G [a, b] then 

h + (u) = hX b (u)h+(b) + h- b (u)h + (a) 

and 

= K b {u)h-{b) + h- b {u)h-{a). 

Solving these equations gives 

+ h-(a)h + (u) - h-{u)h + (a) 

a > b{ > h-{a)h+(b) - h-(b)h+{a) 1 ' 

and 

_ h-(u)h+(b)-h-(b)h+(u) 
n ^ b[U) ~ h-(a)h+(b) - h-{b)h+{a) ■ 1 Uj 

The functions h + and h~ are C 2 on (0, 1) and continuous on [0, 1]. Furthermore, they 

solve Qf = rf where Qf == |<j 2 (-)/ // . In light of this, and remembering our assumption 
that the components of x are ordered, we will look for functions A + and A~ of x\ and x% 
such that 

h(x) = \~(xi,x 3 )h~(x2) + \ + (xi,x 3 )h + (x 2 ) (2-H) 

has the desired properties. For other values of x ^ D we will define h by symmetry. 

To get started, use (I2.9P and f)2.10p so see that f l (x) has a linear dependence on 
/i + (x 2 ) and h~(x 2 ), that is, there are functions ij) % ± such that 



For example, 



f\x) =i£.{xi,x 3 )h (x 2 ) + il} l j r {x 1 ,x 3 )h Jr {x 2 ). 

/i-(xi)/i + (x 3 ) - h-(x 3 )h + (x 1 ) 



ipl(x u x 3 ) 



h~(xx) 



Linearity of the operator (Q l — r) and linear independence of h~ and h + then show 
the requirement that (Q % — r)h = —rf 1 boils down to requiring 

(g'-r) \± = -ni> i ± . 

Of course, the corresponding homogenous (eigenfunction) problems are solved with 
linear combinations of h + and h~ - what remains is the essentially computational task 
of finding particular integrals and some constants. This endeavour begins with repeated 
application of Lagrange's variation of parameters method, determining constants using 
the boundary conditions h(x) = 1 for x G D where possible. Eventually we are left 
wanting only for real constants, an unknown function of X\ and a function of x 3 . At this 
point we appeal to the "smooth pasting" conditions 



_d d_ ]h 

dxi dxj 



0,i,jeV. (2.12) 
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After some manipulation, we are furnished with differential equations for our unknown 
functions and equations for the constants. These we solve with little difficulty and, in 
doing so, determine that 

A_(x l5 x 3 ) = h-( Xl )-h + ( Xl )h + (x 3 ) d l \> du 
+h-(x 1 )h+(x 3 ) / 8 ," 7 \' du 



Jo h~{uf 
2rh-(x 3 ) [ X1 ( h+{u) 



<j> J Q \a(u)h (u) 



2 



(h (xi)h + {u) — h (u)h + (xi)) du, 



and 



r Xl JLh+( U ) 

\+(xx,x 3 ) = h + (x 3 ) + h~ (x{)h~ (x 3 ) I 9 " , N9 du 

Jo h { u y 

+ 2r^_(x 1 ) / MtO V (fe - (M)fe+(a . 3) _ fe - (a . 3)fe+(M))d 
A 3 \cr(u)h+(u)J 

where denotes the constant value of h~(u)-^h + (u) — h + (u)-^h~(u). 

These expressions for A 1 * 1 are valid for any x not lying in D with weakly ordered 
components; so h is defined outside of D via (12. lip . Naturally, we define h to be equal 
to one on D. 

Having defined h, we now show that it is continuous and has the required partial 
derivatives. Continuity is inherited from h + and h~ on the whole of S apart from at 
the exceptional points (0,0,0) and (1,1,1) in D. For these two points, a few lines of 
justification is needed. We shall demonstrate continuity at the origin, continuity at the 
upper right hand corner (1, 1, 1) follows by the same argument. Let x n be a sequence of 
points in S that converge to (0, 0, 0); we must show h(x n ) — > h(0, 0, 0) = 1. Without loss 
of generality assume that the components of x n are ordered < x 1 ^ < x 3 and that x n 
is not in D (if x n G D, then h(x n ) = 1 and it may be discarded from the sequence). By 
examining the expressions for A 1 * 1 , we see that it is sufficient to check that 

(i) \-{x^xf) -> 1 and (ii) h + {x%)\ + {x r l,x™) -> 0. 

For (i), the only doubt is that the term involving the first integral in the expression 
for A - does not vanish in the limit. The fact that it does can be proved by the Dominated 
Convergence Theorem. The term is 

The ratio - j^kj^- 3 ^ is bounded above by one when u > x 3 > x™ since h + is increasing. 
Further, the derivative of h~ is integrable and so the integrand is dominated by an 
integrable function, and converges to zero. 
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For the second limit (ii), there are two terms to check. Firstly, that 



h + (x n 2 )h'(xl)h + (xl 



-du — > 



u 



follows from essentially the same argument as before. The second term of concern is 

h- 



a(u)h+(u) 



(h-(u)h + {xl) - hr{xf)h + {u)) du. 



Again, one may write this as the integral of a dominated function (recalling that a is 
bounded away from zero) that converges to zero. Thus, the integral above converges to 
zero as required. 

Now that we have established continuity of h, we can begin tackling the partial deriva- 
tives. When the components of x are distinct, differentiability comes from that of our 
building blocks h + and h~. It is at the switching boundaries, when two or more com- 
ponents are equal, where we have to be careful. The key here is to remember that we 
constructed h to satisfy the smooth pasting property (12.121) - this allows us to show 
that the one-sided partial derivatives are equal in (0, 1). For example, provided the limit 
exists, 



_d_ 

dxi 



h(x 1 ,x 2 ,x 3 ) 



xi=x 2 =x :j 



lim — (h(xi + h, Xi,X\) — h(x\ 
h^o h 



h,xi,xt)) . 



Using (12. lip and the differentiability of A, the limit from above is 
d 



dx-. 



(A (xi,x 3 )h (x 2 ) + X + (x 1 ,x 3 )h + (x 2 )) 



Xl=X 2 = X 3 



This is equal to the limit from below 





dx\ 



(A (x 1 ,x 3 )h (x 2 ) + X + (x 1 ,x 3 )h + (x 2 )) 



X\=X2=XZ 



by the smooth pasting property.The other first order partial derivatives exist by similar 
arguments. Note that we do not include in our hypothesis the requirement that these 
first order partial derivatives exist at the boundary points of /. 

The second order derivatives are only slightly more laborious to check. As before it is 
at switching boundaries where we must take care in checking that the limits from above 
and below agree. For the partial derivatives -§^ih at a point x not in D with Xi G (0, 1), 

it is continuity of / 1 at x that allows us to equate the limits and show that the result is 
continuous, rather than smooth pasting. For the mixed partial derivatives, a priori, we 
don't have this helping hand. Instead, when two components are equal, we can always 
assume that one is the "middle" component that enters through the the terms h + and 
h~ in (12.111) while the other is an "upper" or "lower" term that enters through A + and 
A~. This makes it easy to check that the partial derivations of h commute at x. Now we 
can use the smooth pasting condition (12.121) to show that, for example, 



d 2 
dx 3 dx 2 



h(x 1 ,x 2 ,x 3 ) 



Xl=X 2 =X 3 



d 2 

dxidx; 



■h(x 1 ,x 2 ,x 3 ] 



xi=x 2 =x 3 
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Thus, h satisfies all the properties we required. 

□ 

From here, we need a verification lemma to check that the function we constructed 
really is equal to v r . The following result does just that, and, as a corollary, shows that 
v r is maximal among Laplace transforms of decision time distributions (note that this is 
weaker than the stochastic minimality claimed in Theorem 11.1 p . The result is essentially 
that Bellman's principle of optimality holds (specialists in optimal control will notice that 
the function we constructed in Lemma [2.11 satisfies the Hamilton- Jacobi-Bellman PDE). 

Lemma 2.2. Suppose that h r : S — > R satisfies 

• h r is continuous on S 

• for i,j E V, q1^ x exists and is continuous on {x E S\D : Xj, Xj E (0, 1)}. 

• h r (x) = 1 for x E D. 

• (G i - r) h r {x) < 
Then, 

h r (x) > supEi (exp(— rr )) . 

c 

Furthermore, if {Q % — r) h r (x) vanishes whenever Xj < Xi < Xk (under some labelling) 
then, 

h r (x) = v r (x) = E x (exp(— rr c *)) . 

Proof. Let C be an arbitrary strategy and define the function g : S x [0, oo) — >• R by 

g{x,t) = f exp(— rt)h r (x). Then, by hypothesis, g is C 2,1 on (0, l) 3 x [0, oo). Thus, if dist 

denotes Euclidean distance and p n = f inf{t > : dist(X c (t), dS) < n^ 1 }, Ito's formula 
shows that, 

g(X c (p n ),p n )-g(X c (0),0) = ^ J £-g(X c (s), s)dXf(s) 

+ J - g (X c (s),s)ds 

Theorem I A. 1 1 implies [Xf] s = [Xj] Ct(s ) and that Xf and Xj are orthogonal martingales. 
Hence, using absolute continuity of C and Proposition 1.5, Chapter V of [T9] . 

^ rpn a 

g(X c (p n ),p n )-g(X c (0),0) = E / g^9(X c (s), s)dXf(s) 

+ V / exp(-rs) (g* - r) h(X c (s))Ci(s)ds. 
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The integrand of the stochastic integral against the square integrable martingale Xf 
is continuous and hence bounded on each compact subset of (0, l) 3 . Thus, the integral's 
expectation vanishes, i.e. 

E x (J* -^g(X c (s),s)dXf(s)) = 
Next, the fact that [Q % — r) h is non-positive gives 

E x ^ exp(-rs) (& - r) h{X c {s))Ci{s)ds \ < 0, 

and so 

Ex (ex P (-rp n )h(X c (p n ))) - h(x) < 0. (2.13) 

Now, the times p n taken for X c to come within distance n~ l of the boundary of S 
converge to p = f inf{t > : X c (t) G dS} as n — > oo. So, the continuity of h and the 
Dominated Convergence Theorem together imply 

Ex (exp(-rp)/i(X c (p))) < h{x). (2.14) 

In summary, inequality (12.141) arises by applying the three dimensional Ito formula to 
g composed with the controlled process stopped inside (0, l) 3 and then using continuity 
of h. But, from time p onwards, our controlled process runs on a face or an edge of the 
cube and Ito's formula in three dimensions does not apply. This is not a problem though 
- a similar argument with Ito's formula in one (or two) dimensions does the trick. That 
is, if p' denotes the first time that X c hits an edge of S (so < p < p' < r c ), then both 

Ex (exp(-rp')/^(X c (p')) - exp(-rp)h(X c (p))) < 0, (2.15) 

and 

Ex (exp(-rr c )h(X c (T C )) - exp(-rp')h(X c (p'))) < 0. (2.16) 

Summing these differences and using the boundary condition h(x) = 1 for x G D 
yields 

Ex (exp(-rr c )) = Ex (exp(-rr c )/i(X c (r c ))) < h(x). 

Thus h is an upper bound for the Laplace transform of the distribution of the deci- 
sion time arising from any strategy. It remains to prove that h is equal to the Laplace 
transform v r . 

Suppose that C is the strategy C* from Lemma 11.41 then for almost every s > 
0, Ci(s) is positive only when Xj(s) < Xff(s) < X%(s) under some labelling. So, 
(Q l — r) h(X c (s))Ci(s) vanishes for almost every s > and (I2.13P is an equality. Taking 
limits shows that (12.141) - (12.161) are also equalities. □ 

So, v r is twice differentiable in each component and satisfies the heuristic equation 

(g i - r) v r (x) = -rf r {x), x £ D, x t G (0, 1). (2.17) 

In the next section we will show that Px(t c > t) is the probabilistic solution to 
certain parabolic partial differential equations. To do this, we need to rewrite v r in a 
more convenient form. 
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It is convenient to introduce the notation = (X x (t), X 2 (0), X 3 (0)), X (2) (t) = 

(X 1 (0),X 2 (2),X 3 (0)) and X^(t) = (Xi(0), X 2 (0), X 3 (t)) for each t > 0. We define p® 
to be the absorption time of Xi, i.e. 



p« d ^ f inf{t>0:X,(t)^(0,l)}. 
Lemma 2.3. For any x ^ D, v r can be written as 

p {i) 



v r {x) = E x exp(-rp w ){) r (X w (p W )) + r / exp(-rs)ds . 



o 



Proof. Fix x ^ D, then the function h- > v r (x) is C 2 on (0, 1) and C° on [0, 1]. Introduce 

the a.s. finite Ti stopping time p$ = f inf{t > : Xi(t) ^ (n _1 , 1 — rir 1 )}, so Ito's formula 
(in one dimension) gives 



exv(-rpV)v r (X«\p®))-v r (X(0)) = / exp(-r S ) — v r (X® (s^dX^s) 



(i) 

Pn Q 



(i) 
fPn 

+ / exp(-rs) (g* - r) v r (X^\s))ds. 
Jo 



The function -^rv r is continuous on (0, 1) and hence bounded on the compact subsets 
[n~ , 1 — n -1 ]. It follows that the expectation of the stochastic integral against dXi 
vanishes. So, using equation (I2.17p . 

v r (x) = E,(exp(-rp«){) r (xW(p«))) 

+rE x {j^ exp(-rs)fUX {i) (s))ds^j . 

The stopping times pn converge to p® as n — > oo and so by continuity of X i} v r , the 
exponential function and the integral, 

exp(-rp«)£ r (X«(pi 4) )) -► exp(-rp«){) r (X»(p«)) and 

exp(-rs)/;(X«(s))ds -> / exp(-rs)ft(X®(s))ds. 
o 7o 

To finish the proof, use the Dominated Convergence Theorem to exchange the limit 

and expectation. □ 

Remark 2.4. We can generalise our heuristic to value functions of the form 

J{x, t) = E x (g(r c * + 1)), x e S, t > 0, 

for differentiable g. It reads 

& + iL) J (*,t) = E x {g'{r c * +t);Ti = 0). (2.18) 



dt 
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Equation (I2.17P is the specialisation g(t) = exp(-rt). Such a choice of g is helpful 
because it effectively removes the time dependence in (12.181) . making it easier to solve. 
The benefit is the same if g is linear and it is not difficult to construct and verify (compare 



Lemmas 12. II and 12. 2j) an explicit expression for J(x) == 'E x (t c *). In terms of the integrals 

T , , def f 3 G(u) , , _ . . def f 1 G(u) , 

h{x\) = / jz \k du and Jk \ x z> = / — ~ du > k E N > 

JO I 1 — u ) Jx 3 u 

the expression for J reads, 

J(x) = G(x 2 ) + (l-x 1 )- 2 G(x 1 )((l-x 2 )((l-x 1 )-(l-x 3 )) + (l-x 1 )(l~x 3 )) 
-2I 3 ( Xl ) ((1 - x 2 )((l - X!) + (1 - a*)) + (1 - ^0(1 - x 3 ))) + 
6/ 4 (xi)(l - x 2 )(l - xi)(l - x 3 ) + xJ 2 G(x 3 ) (x 2 (x 3 - xi) + x x x 3 ) 
-2J 3 (x 3 ) (x 2 (x 3 + xi) + xix 3 ) + 6J 4 (x 3 )xix 2 a; 3 . 

3 A representation for P x (r c * > T) 

The aim of this section is to connect the tail probability v : S X [0, oo) — > [0, 1] defined 
by 

v(x,t) A = { F x {T C * >t), 

to the formula for v r from Lemma 12.31 Before continuing, let us explain the key idea. 
Just for a moment, suppose that v is smooth and consider the Laplace transform of 
[Q % — §i) v(x, •). It is straightforward to show that the Laplace transform of v satisfies 
(see (O), 

poo 

v(t, x) exp(—rt)dt = r _1 (1 — v r (x)) . 



o 



Bringing Q % through the integral and integrating by parts in t, 

exp(-rt) (& - v(x, t)dt = -r- 1 [Q l - r) v r (x). 
Combining this with the heuristic equation (12.171) gives 



o 



exp(-rt) ( G* - ^ ] v(x,t)dt = f l r (x). (3.19) 



This shows that (Q l — J^) v is non-negative, (i.e. v satisfies the associated Hamilton- 
Jacobi-Bellman equation). From here, one could use Ito's formula (c.f. the proof of 
Lemma \2. 21) to see that (v(X c (t),T - t), < t < T) is a sub-martingale for any strategy 
C. In particular, 

F x (r c >T)= Ex {v(X c (T),0)) > v(x,T). 

So, ideally, to prove Theorem 11.11 we would establish that v is smooth enough to 
apply Ito's formula. We are given some hope, by noticing that if we can show that /* is 
the Laplace transform of a function f± say, then (I3.19P implies v solves 

exp(-rt) (gi--)v = fi. (3.20) 



dt 
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We can show such a density /, exists (Lemma 13.11 below) but (surpisingly) not that it is 
Holder continuous. Unfortunately, without the latter we cannot show that (13.201) has a 
classical solution. Nevertheless, we can deduce the sub-martingale inequality by showing 
merely that v solves (13.201) in a weaker sense (Lemma 13.21) . 

To commence, let us first verify the claim that f l r is the Laplace transform of a function. 

Lemma 3.1. For each x ^ D and i G V, the Borel measure B \— > P a (r c G -B,T; = 0) 
has a (defective) density fc-.Sx [0, oo) — > [0, oo), i.e. 

P x .(r c * G dt,Ti = 0) = fi(x,t)dt, t > 0. 

Proof. This is essentially a corollary of the decomposition of r c on {T^ = 0} that was 
discussed in the proof of Lemma 12.11 

Recall that if mi is the first hitting time of level a by Xi (defined in (12.81) ). then for 

x\ < x 2 < x 3 , 

F x (r c * g B, Ti = 0) = P, (mf } + mf } G B, < m®, mf } < mg } ) . 
This is the convolution of the sub-probability measures 

P :E (mS l) G-,mf ) <mg),z = l,2. 
If x\ = x 2 , then T\ > almost surely under f> x , and when x 4. D, x 2 < 1. So, we 

(2) (2) 

may assume that x 2 is in the interval (x\, 1). In this case, {m^ < m^i } is not null and 
X2 can be conditioned, via a Doob /i-transform, to exit (x\, 1) at the upper boundary. 
The conditioned process is again a diffusion and so the arguments of §4.11 of [10] show 

that P x (m 1 ^ G ■,mf' ) < m^j is absolutely continuous. Hence, F x (t c * G -,Ti = 0) is the 

convolution of two measures, at least one of which has a density. 

The other cases are treated with essentially identical arguments. □ 

The next step is to show that v solves (I3.20p in a probabilistic sense. 

Lemma 3.2. Fix i G V and define the function u : S x [0, 00) — > R by 



u(x,t) = Ex I v(XW(t ApW),(t- p W)+)- I f i (X^(s),t-s)ds I , (3.21) 



where = ini{t > : Xi(t) (fc (0, 1)} and zs t/ie density from Lemma \3. 11 Then, 

(a) for each x ^ D ; m(x, •) /ias t/ie same Laplace transform as v (x, ■), 

(b) both u(x, •) and v(x, •) are na/ii continuous, and as a result 

(c) the tail probability v is equal to u and so has the representation given in (I3.2ip . 
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Proof, (a) The Laplace transform of the tail probability is, for x ^ D, 

poo poo 

/ v (t, x) exp(— rt)dt = E x / l[r c *>t] exp(— rt)dt 
Jo Jo 

= E x / exp(-rt)dt 
Jo 

= r _1 (1 - v r {x)) , 

by Fubini's Theorem since the integrand is non-negative. Furthermore, for x e D, both 
t>(t, x) and 1 — v r (x) vanish and so in fact, for any x G S we have 

oo 

v(t, x) exp(-rt)dt = r" 1 (1 - u r (ar)) . (3.22) 



o 



Now, we consider the Laplace transform of u. By linearity of the expectation operator, 



u 



tApW 

(x,t) = E x (v(X®(t A p«), (t - p®)+)) — E x ( / fi(X®(s),t- s)ds 











First consider the Laplace transform of the first member of the right hand side, 

E x (v(X {i \t A p (i) ), (t - p (i) ) + )) exp(-rt)dt. 
Applying Fubini's theorem, the preceeding expression becomes 
Ex (J v(X (i \t A p (i) ), (t - p (i) ) + ) exp(-rt)dt 
which can be decomposed into the sum 



p K -> \ / poo 



W 

E x [ I v(X^(t),0)ex P (-rt)dt\ +Exi^j v(X {i) (p (i) ),t - p (i) ) exp(-rt)dt 



The first term is 

,(0 



E 



/ v(X {t) (t),0)exp(-rt)dt = r~ 1 E x (l -exp(-rp w )) , (3.23) 
Jo 

because when x (£ D, P x -almost surely we have X^(t) ^ D for t < pW. The second term, 

pOO 

Ex / v(X {i \p {i) ),t- p {i) )exp(-rt)dt. 

Jp(i) 

If we shift the variable of integration to u = t—p® and then use (I3.22p . the last expression 
becomes 

r-'Ex (exp(-rp«)(l - v r (X® (p®)))) . (3.24) 
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The treatment of 

J fi(X {i \s),t- s)ds jexp(-rt)dt (3.25) 

proceeds in a similar fashion - exchange the expectation and outer integral and then 
decompose the integrals into t < p^ and t > p^\ The integral over * < p (i) is 

rp (l) rt 

E x / fi(X {l) (s),t- s)dsexp(-rt)dt. 
Jo Jo 

Exchanging the integrals in t and s gives 

Ex / / fi(X®(s),t- s)exp(-rt)dtds. 



For the integral over t > pW, we again exchange the integrals in t and s to give 

/■p^*' roc 

E x / /i(X (i) (s),t-s)exp(-rt)^s. 

Jo J P w 

Summing these final two expressions and substituting u = t — s shows that (13.251) is equal 
to 

rp^ I* oo 

Ex / fi(X^(s),u)exp(-rt)duexp(-rs)ds. 

Jo Jo 

The Laplace transform is a linear operator, and so we may sum (13.231) . (I3.24p and 
(I3.25P to show that the Laplace transform of u is equal to 

r^Ex (l-ex V (-rp®)v r (X®{p®)))+E x ^ exp(-rs)tfcj , (3.26) 

where we have used 

poo 

fi(x, u) exp(-rt)du = f*(x) 



for x D. 

But, (I3.26P is exactly what we get by subsituting the representation for v r from Lemma 
(12. 3p into (13.221) . and so we're done. 

(b) Right-continuity of v in t follows from the Monotone Convergence Theorem. A 
little more work is required to see that u is right-continuous. We begin by observing 
that if pW > t then X{ has not been absorbed by time t and so, if x D, there is a 
P x -negligible set outside of which X®(t) £ D. 

It follows that {X^'(t) E D,p( l > > t} = {pW > t} almost surely. Combining this with 
the fact that v(-,0) = 1[.$d] shows 

E x {v(X®(t A p W ), (t - p«) + ); p« >t)= Fx (p W > t) for x $ D. 

The latter is right-continuous in t by the Monotone Convergence Theorem. The comple- 
mentary expectation 

E x {v(X^\tAp^),(t-p^)+);p^<t) 
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is equal to 




E* / fi(XW(s),t-8)ds 




Using Fubini's theorem we may exchange the integral and expectation to get 



Ex {MX®(s),t-s);p®>s) ds. 



(3.27) 



This suggests the introduction of (p|; s > 0), the transition kernel of Xi killed (and 
sent to a cemetery state) on leaving (0,1). Such a density exists by the arguments of 

§4.11 of jug. 

For notational ease, let us assume i — 1, then (13.271) can be written 



and so regularity of (13.271) in t is inherited from pt. This is sufficient because pj is 
continuous in t > (again see [TO]). 

(c) It follows from (a) that for each x ^ D, u(x,t) and v(x,t) are equal for almost 
every t > 0. Hence, right continuity is enough to show v(x,t) = u(x,t) for every £ > 0. 



From the probabilistic representation for u, we need to deduce some sub-martingale 
type inequalities for t>(X c (i),T — t), < £ < T. As we will see later, it is enough to 
consider strategies that, for some e > 0, run only process during the interval (ke, (k + l)e), 
for integers k > 0. In other words, the rates for each process are either zero or one and 
are constant over (ke, (k + l)e). More specifically, 

Definition 3.3 (e-strategy). For e > we let IT denote the set of strategies C e such that 
for any integer k > 0, 



C e {t) = C e (ke) + (t- ke)£ k , ke < t < (k + l)e, 
where takes values in the set of standard basis elements {(1, 0, 0), (0, 1, 0), (0, 0, 1)}. 



Lemma 3.4. Suppose x G S and < t < T , then the following sub-martingale inequali- 
ties hold. 




Finally, changing the variable of integration from s to s' = t — s gives 




□ 
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(a) For i e V, 

E x {v(X®(t),T-t)) >v{x,T). 

(b) IfC e G IT then 

E x (v(X c \t),T-t)) > v(x, T). 

Proof. Consider first the quantity 

E x (E xW(t) (v(X®((T — t) A p®), (T — t — p«)+))) . (3.28) 

Our Markovian setup comes with a shift operator 9 = for X® defined by 
6 a (u,t) = X {i) (6 s uj,t) = X®(w, s + t) for each u G ft. In terms of this operator, fl3~28|) 
becomes 

Ex (Ex (^(X«((T - t) A p w ), (T — t — P W)+) o . 

From here, use the Tower Property and the fact that p« o9 t = (p w - t) V to find that 
(EEZSH equals 

E x (^(X«(T A p«), (T - p«)+)) . (3.29) 
We can give a similar treatment for 

Ex I E x «(t) ( Jf /.(^(sl.T-i-slrfs 1 . (3.30) 

Using the Markov property of X^\ (13.301) becomes 

Ex I Ex ( J f l (X^(s),T-t-s)dso6 t \T l (t) 

Substituting in for X® o 8 t and p(0 o #t and using the Tower Property, the latter 
expectation is seen to be 

r (T-t)A(p«-i)V0 

Ex I / fi(X®(s + t),T-t-s)ds 



Now make the substitution u = s + t in the integral and use the fact that fi is 
non-negative to show that (13.301) is less than or equal to 

Ex / fi{X®(u),T — u)du I . (3.31) 



The final step is to note that, by Lemma [3.21 

v(x,T-t) = Ex(v(X^(T-tAp^),(T-t-p (l Y)) 

-Ex [J f{X®(s),T-t-s)ds 
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and so E. x (v(X^'(t), T — t)) is equal to (13.281) minus (13.30p . which by the argument 
above is greater than or equal to 

E, (w(X«(T A (T - p w )+)) - E, I j T- u)du\ . 

Again appealing to Lemma [3.21 shows that the latter is exactly v(x,T). 
(b) It is sufficient to prove that for ke < t < (k + l)e we have 

Ex (v(X c \t),T-t)\F c \ke)) >v(X c \ke),T-ke). (3.32) 

The desired result then follows by applying the Tower Property of conditional expectation 
and iterating this inequality. 

Let us take v = C e (ke) and 7i = T^ike). Then v takes values in the grid Z = 
{0, e, 2e, . . .} 3 and A e 7i implies that A = z} is an element of the cr-field T{z) = 
a(J r i(zi), . . . ,.7-3(2:3)) for z G Z. It follows from the definition of conditional expectation 
that f x almost surely we have 

Ex {-\H) = Ex {-\F{z)) on {v = z}. (3.33) 

Now, by continuity of C\ and right-continuity of J-" c \ must be ^-measurable. So, 
if A = f A\ x A 2 x A 3 with Aj Borel measurable for each i e V, (I3.33P gives the equality 

Ex (~fi-[v=z, XC C (t)£A, £_ k =e t ] 1^) = l[v=z,$ k =ei] Ex (l[X(z+(t-ke)ei)eA] \F{z)) , 

where, as before, X(z) = (Xx(zi), X 2 (z 2 ), X 3 (z 3 )). 

Next we use the facts that l[Xj( z ,)eAJ is ^( z ) measurable for each j and that the 
filtration Ti of X, is independent of Tj for j ' ^ i, to show that the preceeding expression 
is equal to 

l[ I /=Z,^=e i ,X,-(z J -)eA J -jyi]E :l: (l^Czi+tt-fce^eAi] |JF . 

Finally, the Markov property of Xj allows us to write this as 

t[u=z^ k =e t ] Ex(z) (l[ X (i)(t-ke)eA]) ■ 

As Ea;(f (XW(t), s)) is Borel measurable for any s, t > 0, this is enough to conclude 
that in our original notation, on = ej}, 

Ex HA ce (t),T - i)|^ CE (A;e)) = Exc E(fce) (^(X«(t - ke),T - t)) . (3.34) 

But part (a) shows that 

Ex (v(X®(t - ke), (T - ke) - (t - ke))) > v(x,T - ke), 

and so the right hand side of (13.341) is greater than or equal to v (X c \ke),T — ke). 

□ 



20 



It is now relatively painless to combine the ingredients above. We take an arbitrary 
strategy C, use Lemma [A. 21 to approximate it by the family C e , e > and then use Lemma 
13.41 part (b) with t = T > to show that 

F x (r c ' >T)=E x v(X c \T),0)>v(x,T) 

for any x ^ D (equality holds trivially for i£fl). 

The approximations are such that C(t) ^ C e (t + Me) for some constant M > 0. Thus, 
r c < t implies that r CE < t + Me. More usefully, the contrapositive is that r c ' > t + Me 
implies t c > t and so monotonicity of the probability measure then ensures 

F x (t c >t)> F x (r ct > t + Me) > v(x, t + Me). 

Taking the limit e — > and using right continuity of v(x,t) in t completes the proof. 

4 Existence and almost sure uniqueness of C* 

In this section we give a proof for Lemma 11.41 Recall that we wish to study strategies C 
that satisfy the property 

(RTM) Ci increases at time t > (i.e. for every s > t, Ci(s) > Ci(t)) only if, under 
some labelling of the processes, 

Xf(t)<Xf(t)<X c k (t). 

Our idea is to reduce the existence and uniqueness of our strategy to a one-sided 
problem. Then, we can use the following result, taken from Proposition 5 and Corollary 
13 in [IS] (alternatively §5.1 of [11J). 

Lemma 4.1. Suppose that (Yi(t);t > 0), i — 1, 2 are independent and identically dis- 
tributed regular Ito diffusions on R, beginning at the origin and with complete, right 
continuous filtrations (7ii(t)\t > 0). Then 

(a) there exists a strategy 7 = (71 (t), 72(^)5 1 > 0) (with respect to the multiparameter 
filtration 7i = (a(7ii(zi), 7^2(^2)); z G R^_) such that 7; increases only at times t > 
with 

i.e. "7 follows the minimum ofY\ and Y 2 " ■ 

(b) If 7' is another strategy with this property, then, almost surely, j'(t) = j(t) for 
every t > 0. That is, 7 is a.s. unique. 

(c) the maximum Y±(t) V Y 2 (t) increases with t. 

We first consider the question of uniqueness, it will then be obvious how C* must be 
defined. Suppose that C is a strategy satisfying (RTM). 

If Xi(0) < X 2 (0) = X 3 (0), then C cannot run X\ (i.e. C\ does not increase) before 
the first time v that either X 2 or X% hit X±(Q). Until then (or until a decision is made, 
whichever comes first), C 2 may increase only at times t > when X 2 (t) < X^(t) and C3 
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only when Xf (t) < X$(t). Hence, on r c A v > t, the value of C(t) is determined by the 
strategy in lemma UT] Now, Xf V Xf increases during this time, and so if v < r c , we 
have 

X^O) = Xf(z/) = X 2 » A Xl(v) < X c 2 {v) V X 3 ». 

So again, we are in a position to apply the argument above, and can do so repeatedly until 
a decision is made. In fact, it takes only a finite number of iterations of the argument 
to determine C(t) for each t > (on r c > t) because each diffusion Xj is continuous, 
the minimum Xf A X 2 A X 3 is decreasing and the maximum Xf V X 2 V X 3 increasing. 
If Xi(0) < X 2 (0) < X 3 (0) then C must run X 2 exclusively until it hits either Xi(0) or 
X 3 (0). From then on, the arguments of the previous case apply. 

The remaining possibility is that Xi(0) = X2(0) = X3(0) = a e (0, 1). We shall define 
random times v e , < e < (1 — a) A a such that 



• C(v e ) is determined by the property (RTM), 

• under some labelling, 

a - e < Xi(u e ) < a < X 2 (u e ) = X%{v e ) = a + e, 

and 

• v e — > as e — > 0. 

Again, we may then use the one-sided argument to see that, almost surely, on u e < t < r c , 
C(t) is determined by (RTM). This is sufficient because v e — > as e — > 0. 

To construct u e , suppose, without loss of generality, that X\ and X 2 both exit (a — 
e, a + e) at the upper boundary. We denote by a, the finite time taken for this to happen, 
i.e. 



Define 



Oi d = inf{t > : Xj(t) ^ (a - e,a + e)}. 



L = inf Xi(s) 



0<S<Qi 

to be the lowest value attained by Xi before it exits (a — e, a + e). By Proposition 5 of 
[T5] . it is almost sure that the Zj are not equal and so, we may assume that I3 < 1% < l\ 
(by relabelling if necessary). 

Intuitively, (RTM) means that Xf and X 2 should hit a + e together while Xf gets 
left down at li- We already know it takes time on for Xj to hit a + e (i = 1, 2) and X3 
takes time 

/? 3 d =inf{t>0:X 3 (t)=/ 2 }. 
to reach Z 2 . So, we set ^ e = ol\ + «2 + fa, and claim that 

C(z/ e ) = (a!,a 2 ,/? 3 ). 

The proof proceeds by examining the various cases. Firstly, if Ci(u e ) > a\ and Ci(u e ) > a\, 
then necessarily £3(14) < /3 3 and X 3 (^ 3 ) > Z 2 for any z 3 < C 3 (z/ e ). But, then there exist 
times a' i < Ci(u e ) (i = 1, 2) with 

l 2 = X 2 (a' 2 ) < X 3 (z 3 ) <X 1 {a[)=a + e 
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for any z 3 < C 3 (i/ e ), contradicting (RTM). 

The second case is that Ci(v e ) < a.\ and C 2 {v t ) < a 2 . Necessarily we then have 
> 03- Now, Xi(zi) > l 2 for Zi < ctj, i — 1, 2 and so (RTM) implies that X 3 (z 3 ) > l 2 
as well for z 3 < C 3 (^ e ). In addition, (RTM) and C 3 (u e ) > (3 3 imply that 

C 2 {v e ) > M{t > : X 2 (t) = l 2 } 

(otherwise X 3 (/3 3 ) < Xi(zi) for z,- t < Ci{v e ), i — 1, 2). So, both X 2 and X 3 have attained Z2 
and then stayed above it for a positive amount of time But, by Proposition 5 in [15] . this 
event that "the lower envelopes of X 2 and X3 are simultaneously flat" has probability 
zero. 

The final case Ci(v e ) > a\ and C 2 (u t ) > a 2 has two subcases, Cs(z/ e ) < j3 3 and Cziy e ) > 
/?3 - both can be eliminated by the methods above. The only remaining possibility is 
that Cj(z/ e ) = «j for i = 1, 2 and C 3 (z/ e ) = /? 3 . 

The discussion above tells us how to define C* - if Xi(0) < X 2 (0) < X 3 (0) under some 
labelling, then we just alternate the one-sided construction from lemma 14.11 repeatedly 
to give a strategy satisfying (CI) - (C3). If X^O) = X 2 (0) = X 3 (0) = a e (0, 1), take 
< e < aA (1 - a) and define C*(i/ U ), < u < e via the construction above. Now, v u 
is only left continuous, so we have yet to define C* on the stochastic intervals (z/ u , u u+ ], 
u < e. But, this is easily done because X c *(z/ U ) has exactly two components equal and 
so we can again use the one-sided construction. We define C* on {y e , t c ] similarly. The 
properties (CI) and (C2) are readily verified. To confirm (C3), we first observe that C* 
satisfies (RTM). But (RTM) gives us almost sure uniqueness of the paths of C*. It follows 
that our definition of C* does not depend on e. The second observation is that v u — > as 
u — > 0. As a consequence, for r\ e and 5 > 0, 

{C*(t) r< v} = {C*(t) ^Vi"u<5 some u < e} 
= \J{C*(t)± V ,v q <5}, 

where the union is over rational numbers < q < e. Using the fact that T is complete, 

{C\t) <r],v q <5}e T{r) X + 5, 772 + 5, % + 6). 

From this we conclude that {C*(t) ^ r/} G J-'i'i]) because T is right continuous. This 
confirms (C3). 

5 x c * as a doubly perturbed diffusion 

We now turn our attention to the optimally controlled process X c * . For convenience, we 
will work with the minimum 

J 4 d = f Xf(t)AXf(t)AX 3 c *(t), 

maximum 

s t d ^xf (t)VXf (t)VXf*(t), 
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and middle value 



M t = (Xf (t) VXf (t)) A (if(t)VXf(f)) A (lf(t)Vlf(t)),t> 

of the components of X c * (so, if Xf(t) < Xf (t) < Xf (t), then I t = Xf*(t),M t = 
X 2 {t),S t = X 3 (t)). There is no ambiguity when the values of the components are 
equal since we are not formally identifying I t , M t and St with a particular component of 
X c * 

Clearly, M behaves as an Ito diffusion solving (11.11) away from the extrema I and 
S, while at the extrema it experiences a perturbation. This behaviour is reminiscent of 
doubly perturbed Brownian motion, which is defined as the (pathwise unique) solution 
(Xp t > 0) of the equation 

X' t = B' t + a sup X' s + (3 inf X'„ 

8<t 

where a, [3 < 1 and (B' t ;t > 0) is a Brownian motion starting from the origin. This 
process was introduced by Le Gall and Yor in [13]; the reader may consult the survey 
[T8] and introduction of [1] for further details. In §2 of 0], this definition is generalised 
to accommodate non-zero initial values for the maximum and minimum processes in the 
obvious way - if i , s > 0, we take 

X[ = B' t + a I sup X' s — s I — (3 I inf X' s + i 

\8<t J V s -* 

i.e. X' hits — io or so before the perturbations begin. As usual a + = max(a, 0) and 
a - = max(— a, 0). 

Our suspicion that M should solve this equation if the underlying processes are Brow- 
nian motions is confirmed in the following 

Lemma 5.1. Suppose that < io < m < s < 1 and a — 1. Then, under P(i , m0) s ); 
there is a standard Brownian motion (B' t ;t > 0) (adapted to T c * ) for which the process 
M' = M t - mo, t > satisfies 

M't=B' t - (swp M' s - A +(ig t M' s + i' 

where i' = mo — io and s' = so — mo- In other words, M is a doubly perturbed Brownian 
motion with parameters a = (3 = — 1. 

Proof. The multiparameter martingale (X\(z\) + X2{z2) + X^{z^); z e M? + ) is bounded 
and right continuous. Hence, Theorem I A. II implies that 

it = xf (t) + Xf (i) + X 3 C * (i ) , t > 

is a continuous (single parameter) martingale with respect to the filtration T c * . But, 
the Xj are independent Brownian motions and so the same argument applies to the 
multiparameter martingale 

((Xi(zi) + X 2 {z 2 ) + X 3 (z 3 )) 2 - { Zl + z 2 + z 3 ); z G , 
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i.e. £ 4 2 — t is a martingale. It follows that (&; t > 0) is a Brownian motion with £o = 
z'o + mo + so and we can take 5' = £ — (io + rrio + so). 

Now, C* always "runs M" away from the extrema I and 5* of X c and so it is no 
surprise that 

I t = inf M s A i , S t = sup M s V s , 
relationships which can be proved using the arguments of section HI It follows that 
M[ = M t - m = £t - m - S t - I t = B' t - sup M s V s + s - inf M s A i + «o 



The result now follows by noting that for real a and b we have a Ab — b = —{a — b) 
and a V 6- b= (a -b) + . □ 

Lemma I5TT1 is relevant because t c * is precisely the time taken for the doubly perturbed 
Brownian motion M to exit the interval (0, 1). In particular, the expression we find for 
the Laplace transform v r (x) can be recovered from Theorems 4 and 5 in Chaumont and 
Doney [3]. 

We have so far assumed that a = 1 and are yet to say anything about more general 
"perturbed diffusion processes". There are several papers that consider this problem. 
Doney and Zhang [5] consider the existence and uniqueness of diffusions perturbed at 
their maximum. More recently, Luo [H] has shown that solutions to 



X' t = [ / j(s J X' s )ds+ [ a(s,X' s )dB' s + a sup X' s + (3 inf X' s , 

JO Jo s<t s <* 



(5.35) 



exist and are unique, but only in the case that \a\ + \/3\ < 1. A more general perturbed 
process is considered in [9] but similar restrictions on a and (5 apply. 

That is, there are no existence and uniqueness results for doubly perturbed diffu- 
sions which cover our choice of a and [3, and less still for the Laplace transform of the 
distribution of the time taken to exit an interval. 

This is where our results seem to contribute something new. Lemma 15.11 easily gen- 
eralises to continuous a > 0, and this combined with the other results in this paper, lets 
us see that if /i is bounded and Borel measurable and a > is continuous, then there is 
a solution to 

M[= [ fi{M' s )dB' s + [ a(M' s )dB' s -supM s -infM s . 

Jo Jo s<t s <* 

Furthermore, we can compute the Laplace transform of the distribution of the time taken 
for any solution of this equation to exit any interval (—a, b) when /i is zero. 



6 Concluding remarks and future work 

6.1 Majority decisions of 2k + 1 diffusions and veto voting 

The problem that we have solved has natural generalisations in which there are m diffu- 
sions instead of the three that we have considered. 
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In particular, one might ask for the majority decision of an odd number of 'diffusive 
voters' (Xi(t);t > 0), i = 1, ...,m. Again, we believe that the optimal strategy is to 
"run the middle". In other words, if m = 2k + 1, and 

xT(t) <...< x c k \t) < xi* +1 (t) < x c k * +2 (t) < . ..x%(t) 

then C\ should increase at unit rate until X%* +1 hits either X% (t) or X% +2 (t). This 
prescribes that until then, all other components of X c are constant. 

A special case of majority voting is 'veto voting', where we have an arbitrary number 
ml > of diffusions, and declare a negative decision if at least k < ml of them get 
absorbed at the lower boundary (otherwise no veto occurs and a positive decision is 
made). To see that this is a majority voting problem, suppose that there is no veto if the 
majority of voters return positive decisions (i.e. 2k < ml). This is equivalent to asking 
for a majority of m — 2{ml — k) + 1 diffusive voters, with m + 1 — 2k of them beginning 
in a state of absorption at the origin. The case 2k > ml admits a similar majority voting 
description and in particular, our conjecture for veto voting is that if 

x?(t) <...< x^S) < x c k \t) < x c k * +1 (t) < . ..xZ(t) 

then C k should increase at unit rate until X k * hits either X^^it) or X k * +l {t). In other 
words, we "run the component with k th order statistic" . The extreme of this is true veto 
voting in which a single diffusion being absorbed at zero will veto the others. This is 
the case k — 1, and the conjecture is that we should always "run the minimum" of the 
diffusions. 

One might also consider diffusions which obey different stochastic differential equa- 
tions. We have found an implicit equation for the switching boundaries in the optimal 
strategy for ml = 2, k = 1 'veto voting' problem by solving a free boundary problem. 
However, we have no conjecture for the general solution. 

6.2 Recursive majority revisited 

To close, we return to the discrete recursive majority model that motivated us originally 
(see discussion in the introduction). Recall that r n denotes the expected cost of the 
optimal strategy for the n layer tree. The best lower bound for the limit 7 = lim^oo r\[ n 
in the literature^ is |, which one arrives at by computing the Fourier coefficients of the 
recursive majority function and applying either an equality due to O'Donnell and Servedio 
(see §3 of [TTj ) or Theorem 1.8 of [20]. But, numerics suggest 7 ~ 2.45, leaving big a gap. 
The best upper bound known to us is 7 < 2.472. 

In the intrduction, we hinted at a continuous approximation to the discrete tree. What 
we had in mind was to replace each of the Bernoulli random variables on the leaves with a 
Brownian motion starting from p. These Brownian motions are absorbed at the endpoints 
of (0, 1) and scaled so that the expected absorption time is one. As with the diffusion 
model treated in this paper, the observer is billed for the time they spend running each 
Brownian motion. Let R n denote the least expect (time) cost for the Brownian tree. In 

1 It has been communicated to us that Oded Schramm and Mark Braverman improved this bound to 
2.28 but did not publish details. 
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this paper (see Remark 12. 4p . we have shown 

Rl &) = ~ p(l-p) ~ti +p2 + ~ M 1 - p ^ ' 

so Ri<n = 2(l+p(l-p)). 

Now, any strategy for the discrete model can also be used in this Brownian case and 
so R n is not greater than r n . It follows that 

7 > limsupi?y n , 

n—*oo 

from which we conclude that studying the Brownian tree may help give a lower bound 
for 7. 

Often, the Brownian version of a difficult discrete problem is easier to solve because 
we have the heavy machinery of stochastic calculus at our disposal. But, we concede 
that there is no particular reason to think that the n layer Brownian model may be more 
tractable than the discrete counterpart. Indeed, while we have treated the n — 1 case in 
this paper, we are unable even to give a conjecture on the n = 2 optimal strategy. 

Still, we might ask, even if it is not possible to determine the optimal strategy, can 
we say anything about the asymptotics of the expected cost R n , and in doing so sharpen 
the bound on 7? We have not, for example, been able to prove that Rn is eventually 
monotone in n. Nor do we have the sub-multiplicative structure to guarantee that V = 
lim^oo Rl! n even exists. 

If the limit V does exist, is it equal to 7? One is tempted to guess affirmatively but 
it is possible that the optimal strategy runs an exponentially growing number of leaf 
Brownian motions for very short time, leading to V < 7. To us at least, this seems a 
tough question to answer. 



A A result from the theory of multiparameter pro- 
cesses 

The proofs of Lemmas 12.21 and 15 . 1 1 appealed to the fact that a multiparameter martingales 
composed with a strategy is again a martingale. Moreover, it was asserted that we can 
approximate an arbitrary strategy with a discrete one. This appendix contains a precise 
statement of these results, together with basic definitions (adopted from §4 of [7]). 

Let (fl, J 7 , P) be a complete probability space, M + denote the set of non-negative reals 
[0,oo) and d > 2. A family (^(77); e M d ) of a-algebras contained in T is called a 
multiparameter filtration if, for every rj, v G R d > with 7] H u, 

It is assumed that the following "usual conditions" hold; 

• Fiji) = F{y) for every r\ G IR+ (right continuity) 

• ^"(0) contains all null sets (completeness) 
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• for any 77, v G R+, the a-algebras Fiyj) and T{y) are conditionally independent 
given T{r\ A v). 

The final condition (usually referred to as "assumption (F4)" after (2j) is trivially 
satisfied in our case since JF is generated from independent nitrations. 

A real valued process [Z{r\); 77 G Mi) is called a multiparameter super-martingale with 
respect to (J-{rj)\ 77 G M d ) if for every 77, 

• E 1^(57)1 < 00, i.e. Z is integrable, 

• Z(rj) is J-(ri) measurable and 

• E(-Z'(^)|^ r (^)) < for every 77 -< v. 

Recall that a strategy C is a IR+ valued process such that Cj increases from the origin, 
= t and {C(t) ■< 77} G .Ffa) for every t > and 77 G (conditions (C1)-(C3) 
from section [T^]) . Then, the filtration [T (t);t > 0) defined by 

f c (t) = {F6f:Fn {C(t) r< 77} G ^(77) V77 G M^}, t > 

satisfies the usual conditions. The process 

Z cd ^(Z 1 (C 1 (t)),...,Z (i (C d (t));t>0) 

is adapted to this filtration. 

The idea is that Z c should be a super-martingale with respect to T c . Indeed, Propo- 
sition 4.3 in [7] reads 

Theorem A.l. Suppose that Z is a right continuous multi-parameter super-martingale 
and that C is a strategy. Then Z c is a (local) T c -super-martingale. 

This theorem appears in various guises throughout the literature (a good reference for 
the discrete case is chapter 1 of [T]), we do not give the proof. Merely, we will mention 
one of its stepping stones - approximation of an arbitary strategy with a discrete one. 

Recall from definition 13.31 that for any e > 0, TI e denotes the set of strategies which 
only increase in one component over each interval [ke, (k + l)e), k = 0,1, . . ., i.e. C e is in 
il e if d a.e. takes only values or 1 and is constant on each interval [ke, (k + l)e). The 
promised approximation result is 

Lemma A. 2. For any strategy C, there exist a family of strategies C e G IT, e > that 
converge to C in the sense that 

limsup|C(t) -C e \ = 0, 

where | • | is any norm on M d . 

Moreover, there is a positive constant M > for which C(t) z< C e (t + Me) for every 
t > 0. 

The existence and uniform convergence part of this lemma is exactly Theorem 7 of 
Mandelbaum [15] and the second part is a corollary to the author's constructive proof. 
The details are omitted. 

Corresponding author: Peter Windridge p.windridge@warwick.ac.uk 
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